The invention relates to a method and a system for displaying dynamic safety-relevant three-dimensional contents on a display device, in particular on a display device of a vehicle.
Vehicles include microprocessor-controlled systems on which applications which generate three-dimensional (3D) image data are executed. To this end, in the prior art, each application builds a separate so-called scene model that describes a three-dimensional scene. So-called renderers are used to represent the three-dimensional scene on a display unit. Said systems can likewise be executed on a microprocessor, in particular on a computer. They are substantially used to process the three-dimensional image data of the three-dimensional scene such that they are adapted for representation on the display unit.
During a rendering process, a two-dimensional image can be calculated from a three-dimensional scene, for example. During the conversion of three-dimensional image data which describe a vector graphic, it is possible in the course of the rendering process, for example, to convert the vector graphic of an object into an image-point-wise pixel representation of the object in a two-dimensional (2D) computer graphic.
A three-dimensional renderer can generate, for example, a separate two-dimensional graphic from each individual three-dimensional scene. A control component, a so-called layer manager, can be used to produce a total image for the display on a display unit by superimposing different two-dimensional graphics. Here, the individual two-dimensional images are placed one above the other according to a fixed sequence in the form of planes. Contents from a higher plane can here cover contents from a lower plane. For the topmost plane, the visibility of its contents can be guaranteed.
Owing to such an architecture or data processing based on planes, three-dimensional contents of different applications can be displayed on a common display. It is also possible to ensure here that contents of a safety-relevant application are represented on the display, that is to say that they are not covered by contents of other, non-safety-relevant applications.
Safety aspects are an important consideration in displays in a vehicle. Two-dimensional contents are currently predominantly represented on a display device in a vehicle. In order to ensure that these contents are displayed correctly, they are stored in a special hardware area and are displayed above all other contents. Safety-relevant displays are, for example, a warning of an excessively low oil level, a warning of an incorrect tire pressure and/or a warning of restricted functionality of the brake.
The representation of three-dimensional contents causes interaction between the contents which include, for example, lighting effects, reflection, shadowing and the like. These contents cannot be statically stored like the two-dimensional contents but rather must be calculated at the run time. A graphics processor (GPU) is needed to efficiently calculate the representation of the three-dimensional contents on a display device which represents a two-dimensional image. A graphics processor is generally not suitable for representing safety-relevant contents since they are not designed and certified for that purpose.
U.S. Pat. No. 8,286,196 discloses that programs can be alternatively executed by a central processor unit (CPU) or by a graphics processor (GPU).
DE 10 2013 201 377.9, owned by the Applicant of the present application and filed on an earlier date, discloses a method and an image processing system which at least partially superimposes three-dimensional image scenes and forms a three-dimensional overall scene. Three-dimensional output image data are also rendered. The subject matter of DE 10 2013 201 377.9 is hereby incorporated by reference.
The invention is based on the object of providing an improved method and system for displaying three-dimensional contents on a display device of a vehicle which represents three-dimensional contents in a two-dimensional manner.
The object of the invention is achieved by a method and a display system according to embodiments of the invention.
The method according to the invention for representing an item of three-dimensional graphical information on a display device of a vehicle comprises the act of calculating a first item of two-dimensional graphical information for at least one first area of the display device from the three-dimensional information using a graphics processor, the display device representing the three-dimensional information in a two-dimensional manner. According to the invention, a second item of two-dimensional graphical information is calculated for at least one second area of the display device from the three-dimensional information using a universal processor, the second area being part of the first area. A check is carried out in order to determine whether the first and second items of two-dimensional graphical information have a degree of correspondence which is higher than a threshold value. The first item of two-dimensional information is output on the display device if the first and second items of two-dimensional graphical information have a degree of correspondence which is higher than the threshold value.
The operation of generating the first item of two-dimensional graphical information and the operation of generating the second item of two-dimensional graphical information can be carried out using rendering. The universal processor may be a central processor unit (CPU). The first and second areas may be of the same size. The universal processor checks the method of operation of the graphics processor. If the degree of correspondence is lower than a threshold value, it is possible to output a warning indicating that the contents have been manipulated or there is a malfunction of a program.
The act of checking whether the first and second items of two-dimensional graphical information have a degree of correspondence which is higher than a threshold value can be carried out by a comparison device which comprises neither the graphics processor nor the universal processor. This makes it possible to further reduce the risk of manipulation. However, it is also possible for the comparison device to be implemented by use of software which runs on the universal processor.
The term “a universal processor” should be interpreted in such a manner that there is at least one universal processor. The term “a graphics processor” should also be interpreted in such a manner that there is at least one graphics processor. The at least one graphics processor and the at least one universal processor need not be situated in the same calculation device, for example a control unit or the like. They may be distributed over the vehicle and may be situated in different calculation devices which are connected via a network. The method consequently comprises the act of calculating the first item of two-dimensional graphical information using a plurality of graphics processors. The method also comprises the act of calculating the second item of two-dimensional graphical information using a plurality of universal processors.
The reliability of the displayed contents is achieved by way of the redundancy. The likelihood of the results from two completely different hardware and software architectures providing the same error is sufficiently low. The graphics processor has a high degree of parallelization when calculating the data in the shader units, while the universal processor has a very low degree of parallelization in comparison with the graphics processor. The software of the two processor types differs considerably as the operations of the graphics processor are controlled by a driver provided by the manufacturer and the calculations on the universal processor are carried out using separately developed software.
The at least one universal processor can calculate the second item of two-dimensional graphical information in a secured mode and/or security mode in which the data are protected against manipulation.
The method may determine a safety-relevant area of the three-dimensional information and may stipulate the second area in such a manner that it comprises the safety-relevant area. A check can then be carried out in order to determine whether the first and second items of two-dimensional graphical information have a degree of correspondence which is higher than the threshold value. This increases the efficiency since only the safety-relevant area is checked.
The first and second items of two-dimensional information may include a lighting effect, reflection and/or shadowing which are calculated on the basis of the three-dimensional data. As a result, the three-dimensional representations affect the observer like a natural scenario.
The three-dimensional information can be calculated at the run time. It is undesirable for the three-dimensional representations to be permanently stored in a hardware memory since no individual scenarios can be created as a result. The three-dimensional information can be generated by a plurality of processes running on the same calculation device, for example a computer, a control unit or the like. However, it is also possible for the three-dimensional information to be generated by a plurality of processes running on different calculation devices, for example computers, control devices or the like. The calculation devices may be connected via a network and/or a bus.
The three-dimensional information may have a first item of three-dimensional partial information which is assigned a first priority and is represented in front of or above a second item of three-dimensional partial information which is assigned a second priority which is lower than the first priority. A plurality of distributed processes may represent information on the display device. For example, it is possible for a first process to represent the image from a rear-view camera on the display device. A second process may represent the predicted trip trajectory on the basis of the steering angle. A third process may evaluate the information from the distance sensors and may indicate a possible risk of collision. This information may be represented above one another on the display device.
If it is determined, while representing the image from the rear-view camera, the predicted trajectory and the distance information, that the oil level is too low, the information indicating that the engine should be immediately switched off and the oil level should be checked is displayed above the image from the rear-view camera. In this example, the oil level information has the highest priority, the distance information has the second-highest priority, the predicted trajectory has the third-highest priority and the image from the rear-view camera has the lowest priority.
The three-dimensional information may include an effect description. An effect description describes a three-dimensional graphic to be represented in an abstract manner. The effect description includes a platform-independent description of the manner in which a graphics processor is intended to display the individual graphical elements. In terms of content, the effect description is substantially identical to a so-called shader code but is not dependent on a particular graphics API (for example OpenGL or DirectX). The description additionally includes the configuration of the graphics processor which goes beyond the shader code, for example the blend mode, the depth test, etc.
The data format for this effect description is a tree structure. This resembles an abstract syntax tree, as is also used by compilers. The effect description corresponds substantially to the result after a parser has processed the source code.
The operation of generating the two-dimensional information may also include time-slice rendering. The problem often exists that contents of different dynamics are situated in a scene, for example a speedometer needle which requires a very high degree of responsiveness and a navigation scene which is less time-critical. It would be possible either to render the navigation and the speedometer needle with a very large number of frames or to render both with a smaller number of frames. The practice of rendering the navigation with the number of frames required for the speedometer needle usually overwhelms the hardware. In contrast, the practice of rendering the speedometer needle with the small number of frames for the navigation causes non-fluid animation of the speedometer needle. So-called time-slice rendering is used in such constellations. In this case, the speedometer needle is rendered using a high frame rate (for example 60 fps) and a portion of the navigation is concomitantly rendered in each frame but is not displayed. Instead, an older image is used for the navigation. If it is assumed that 20 fps are intended to be achieved for the navigation, only one third of the navigation scene is concomitantly calculated in each frame. A complete frame for the navigation is then also available after three frames for the speedometer needle. In this case, the speedometer needle can be blended into the new navigation image and a complete image is obtained again. This technique makes it possible to better distribute the load.
The invention also relates to a computer program product which, when loaded into a computer having a processor and a memory, carries out the steps of the method described above. The computer may be a control unit.
The invention also relates to a display system which is designed to represent an item of three-dimensional graphical information on a display device in a vehicle. The display system comprises a display device which represents an item of three-dimensional information in a two-dimensional manner, at least one graphics processor, at least one universal processor, a coordination device and a comparison device. The coordination device instructs the graphics processor to calculate a first item of two-dimensional graphical information for at least one first area of the display device from the three-dimensional information. The coordination device instructs the universal processor to calculate a second item of two-dimensional graphical information for at least one second area of the display device from the three-dimensional information. The coordination device instructs the comparison device to check whether the first and second items of two-dimensional graphical information have a degree of correspondence which is higher than a threshold value.
The at least one graphics processor, the at least one universal processor, the control device and the comparison device may be situated in a calculation device or may be distributed among a plurality of calculation devices. The control device and/or the comparison device may be implemented by way of the universal processor or by way of dedicated hardware or a dedicated device. After it has been determined that the first and second items of two-dimensional graphical information have a degree of correspondence which is higher than the threshold value, the first item of two-dimensional graphical information can be output on the display device.
The display system can be developed in the manner described above with respect to the method.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of one or more preferred embodiments when considered in conjunction with the accompanying drawings.